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Carretta  TR.  U.S.  Air  Force  pilot  selection  and  training  methods. 
Aviat  Space  Environ  Med  2000;  71:950-6. 

Background:  U.S.  Air  Force  (USAF)  pilot  selection  and  training  pro¬ 
cedures  have  changed  substantially  in  the  last  several  years.  Pilot  selec¬ 
tion  procedures  were  augmented  with  the  introduction  of  computer- 
based  testing  to  assess  pilot  aptitude.  Training  procedures  have  changed 
in  an  effort  to  modernize  the  training  fleet  and  provide  better,  more 
specialized  training  earlier  in  the  training  process.  Results:  Despite 
several  studies  showing  the  utility  of  USAF  pilot  aptitude  tests  for  pre¬ 
dicting  training  performance,  results  of  a  recent  policy  capturing  study 
suggest  that  this  information  is  often  ignored  by  pilot  candidate  selection 
boards.  The  two  largest  sources  of  USAF  pilot  trainees  relied  heavily  on 
measures  of  officership  when  making  selection  decisions.  Cumulative 
research  findings  suggest  that  USAF  pilot  selection  decisions  could  be 
improved  by  making  better  use  of  currently  available  personnel  attribute 
data.  Further  improvements  could  be  expected  from  the  addition  of  a 
structured  selection  interview  and  measures  of  personality. 
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The  keyword  in  describing  U.S.  Air  Force 

(USAF)  pilot  selection  and  training  over  the  last  5  yr 
has  been  "change."  Training  procedures  have  changed 
in  an  effort  to  modernize  the  training  fleet  and  provide 
better,  more  specialized  training  earlier  in  the  training 
cycle  (20).  Pilot  selection  procedures  have  changed  to 
take  advantage  of  computer-based  testing.  The  purpose 
of  this  paper  is  to  review  these  changes. 

Changes  in  USAF  Pilot  Training 

Undergraduate  Pilot  Training  (UPT) 

In  Undergraduate  Pilot  Training  (UPT),  pilot  trainees 
completed  primary  jet  training  in  the  T-37  (21  wk),  then 
moved  on  to  advanced  jet  training  in  the  T-38  (32  wk) 
(see  Fig.  1).  Near  completion  of  T-38  training  an  ad¬ 
vanced  training  recommendation  board  consisting  of 
T-38  instructor  pilots  evaluated  student  records  and 
made  recommendations  for  post-UPT  specialized  train¬ 
ing  assignments  in  either  tanker-transport-bomber 
(I^)  aircraft  or  fighter-attack-reconnaissance  (FAR) 
aircraft.  Specialized  training  assignments  were  based 
on  instructor  recommendations,  student  preferences, 
and  aircraft  availability. 

Specialized  Undergraduate  Pilot  Training  (SUPT) 

UPT  was  phased  out  in  the  mid-1990s  and  replaced 
with  Specialized  Undergraduate  Pilot  Training  (SUPT). 
Under  SUPT,  student  pilots  complete  a  common  T-37 
training  phase  (as  in  UPT).  At  the  end  of  T-37  training, 
students  are  assigned  to  one  of  four  advanced  training 
tracks.  Most  are  assigned  to  either  the  bomber/ fighter 
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Fig.  1.  UPT  and  SUPT  Pipelines. 

(B/F)  or  airlift/ tanker  (A/T)  tracks.  A  few  are  assigned 
to  the  T-44  advanced  turbo-prop  or  UH-1  helicopter 
tracks.  Track  assignments  are  based  on  T-37  training 
performance,  aircraft  availability,  and  student  prefer¬ 
ence.  B/F  training  (12)  takes  place  in  the  T-38  and  A/T 
training  (13)  takes  place  in  the  T-IA,  a  modified  Beech 
400  business  jet.  The  T-1  has  an  impressive  electronic 
flight  instrumentation  system  with  pilot  configurable 
multifunction  displays,  flight  management  system, 
weather  radar,  and  electronic  displays  of  maps  and 
checklists. 

The  reasons  for  the  change  to  SUPT  included  the 
desire  to:  a)  prolong  the  life  of  the  T-38;  b)  upgrade  the 
training  fleet;  and  c)  improve  training.  As  not  all  stu¬ 
dents  would  need  to  go  through  T-38  training  (as  in 
UPT),  the  life  span  of  the  T-38  could  be  extended.  A 
shortcoming  of  the  generalized  flight  training  provided 
in  UPT  was  that  it  did  not  provide  student  pilots  with 
specialized  skills  and  knowledge  needed  to  smoothly 
transition  to  the  large  transport  and  tanker  aircraft  that 
constitute  one-third  of  the  USAF  fleet.  With  the  imple¬ 
mentation  of  T-1  training,  A/T  students  could  receive 
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specialized  instruction  at  an  earlier  stage  in  their  train¬ 
ing. 

Recently,  the  USAF  modified  the  training  sequence 
for  bomber  pilots.  For  a  short  period  starting  in  late 
1998,  B-1  and  B-52  bomber  pilots  were  selected  from 
among  T-1  trained  students  effectively  making  the  T-38 
track  a  "fighter  only"  track.  More  recently,  the  policy 
has  changed  again  where  B-1  pilots  are  selected  from 
among  T-38  students  and  B-52  pilots  from  among  T-1 
students. 

More  Changes  to  Come 

The  Air  Force  goal  of  upgrading  the  trainer  fleet  with 
current  technology  aircraft  has  resulted  in  decisions  to 
replace  the  aging  T-37  and  to  upgrade  T-38  avionics  to 
contemporary  standards.  The  aircraft  chosen  to  replace 
the  T-37  is  the  Beech  Mk  II,  known  as  the  Texan  II  (2). 
Student  pilot  training  in  the  Texan  II  is  expected  to 
begin  in  2001.  The  aircraft  will  eventually  replace  all 
T-37s  in  SUPT  and  Euro-NATO  Joint  Jet  Pilot  Training 
(ENJJPT).  The  flight  training  syllabus  for  the  Texan  II 
will  resemble  that  used  today  for  T-37  primary  training, 
similar  in  duration,  flight  hours,  and  training  events  (3). 

Modifications  to  the  T-38  will  allow  the  USAF  to 
include  mission  specific  weapons  delivery  and  tactics 
training  in  the  fighter  track.  A  major  avionics  upgrade 
will  replace  existing  T-38  instrumentation  commencing 
in  1999  (1).  By  2008,  all  T-38  A  and  T-38B  aircraft  used  in 
SUPT  and  ENJJPT  will  be  upgraded  to  T-38C  configu¬ 
ration. 

In  addition  to  the  T-38  hardware  upgrade,  current 
plans  include  extending  fighter  track  training  by  7  wk 
to  incorporate  training  now  provided  in  a  separate 
fighter  fundamentals  course.  This  potential  change  is 
not  likely  to  occur  until  the  T-38C  modification  has  been 
fully  integrated  into  the  force. 

Flight  Screening  Program 

The  pre-SUPT  Flight  Screening  Program  (FSP)  also 
has  changed  significantly.  Prior  to  1994,  FSP  consisted 
of  about  14  h  instruction  in  the  T-41.  In  1994,  the  T-41 
was  replaced  with  the  T-3  (Slingsby  Firefly)  and  the 
hands-on  training  was  extended  to  21  h.  This  Enhanced 
Flight  Screening  Program  (EFSP)  included  more  aero¬ 
batics  training  and  was  popular  with  students  and  in¬ 
structors.  However,  in  July  1997  the  T-3  flight  opera¬ 
tions  were  suspended  following  several  uncommanded 
engine  stoppages  (4). 

With  the  suspension  of  T-3  training,  many  students 
were  arriving  at  SUPT  with  no  hands-on  flying  experi¬ 
ence.  This  placed  a  greater  burden  on  T-37  instructors  to 
get  student  pilots  "up  to  speed."  Even  students  who 
had  prior  flying  time  in  commercial  aircraft  were  some¬ 
times  having  problems  adapting  to  military  flight  pro¬ 
cedures.  Without  the  benefit  of  T-3  flight  screening, 
attrition  rates  for  SUPT  climbed  above  15%  (4).  The 
USAF  considers  an  8-10%  attrition  rate  acceptable. 

As  a  result  of  the  perceived  need  for  pre-SUPT  flying 
training,  in  October  1998  the  USAF  implemented  an 
Introductory  Flight  Training  (IFT)  program.  The  IFT 
program  included  up  to  40  h  flying  time  in  commercial 


aviation  training  programs  and  required  a  solo  flight. 
The  40-h  IFT  program  reportedly  has  produced  similar 
attrition  rates  in  the  T-37  phase  of  SUPT  as  were  ob¬ 
served  when  EFSP  was  in  use  (IFT — 8.8%  vs.  EFSP — 
7.8%)  (4).  In  October  1999,  the  USAF  announced  plans 
to  implement  an  expanded  IFT  program  (4).  The  ex¬ 
panded  program  will  increase  hands-on  flying  time  to 
50  h  and  will  require  additional  solo  flights  and  stu¬ 
dents  to  earn  a  private  pilot's  license.  In  the  expanded 
IFT  program,  students  will  receive  Federal  Aviation 
Administration-certified  flight  instruction  through  local 
flight  schools.  More  than  150  flight  schools  nationwide 
may  be  involved  in  the  training  program,  mostly  near 
colleges  and  imiversities  with  Air  Force  Reserve  Officer 
Training  Corps  (AFROTC)  detachments.  Pilot  candi¬ 
dates  at  the  U.S.  Air  Force  Academy  will  receive  their 
training  in  the  local  area. 

After  completing  the  expanded  IFT  program,  pilot 
candidates  will  enter  SUPT.  Although  T-37  attrition 
rates  have  been  similar  following  IFT  as  were  observed 
following  EFSP,  some  have  questioned  its  utility  for 
USAF  training  and  screening.  Skeptics  note  that  IFT  is 
not  done  in  a  military  context  (as  was  EFSP)  and  em¬ 
phasizes  training  only,  not  training  and  screening. 

USAF  Pilot  Selection  Methods 

U.S.  Air  Force  pilot  qualification  standards  include 
medical  fitness,  anthropometric  standards,  educational 
achievement  (e.g.,  college  grade  point  average,  major), 
and  for  some  commissioning  sources,  aptitude  test 
scores  and  successful  performance  in  a  flight  screening 
program  (19).  The  indicators  of  pilot  aptitude  that  are 
used  vary  by  source  of  commission.  Further,  although 
pilot  aptitude  test  scores  are  used  to  express  qualifica¬ 
tion  minimums  for  some  points  of  entry,  the  manner  in 
which  these  scores  are  used  by  selection  boards  varies 
(18,19). 

Pilot  Aptitude  Tests 

The  two  most  widely  used  pilot  aptitude  tests  in 
the  USAF  are  the  Air  Force  Officer  Qualifying  Test 
(AFOQT;  9, 10)  and  the  Basic  Attributes  Test  (BAT;  6, 
8).  The  AFOQT  is  a  16-test  paper-and-pencil  multiple 
aptitude  battery  used  for  officer  commissioning  and 
aircrew  selection.  It  measures  general  cognitive  abil¬ 
ity  (^),  verbal,  math,  spatial,  aviation  knowledge,  and 
perceptual  speed  (10).  The  AFOQT  has  been  in  use 
since  1957  with  new  forms  developed  about  every  7 
yr.  The  16  tests  are  combined  into  five  composites: 
Verbal,  Quantitative,  Academic  Aptitude  (Verbal  + 
Quantitative),  Pilot,  and  Navigator-Technical  (9,10). 
The  AFOQT  Pilot  composite  consists  of  eight  tests 
that  measure  knowledge  of  aviation  and  mechanical 
systems,  the  ability  to  determine  aircraft  attitude 
from  flight  instruments,  knowledge  of  aeronautical 
concepts,  ability  to  read  and  interpret  scales  and 
tables,  and  spatial  ability. 

The  BAT  is  a  five  test  computer-based  battery  used 
solely  for  pilot  selection.  It  measures  cognitive  ability, 
psychomotor  ability,  and  attitudes  toward  risk.  Pre- 
operational  forms  of  the  BAT  included  several  addi- 
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TABLE  I.  AFOQT  MINIMUM  QUALIFYING  SCORES  FOR  PILOT 
SELECTION  BY  COMMISSIONING  SOURCE. 


Source 

Minimum  AFOQT  Composite  Scores 

Pilot 

Nav- 

Tech. 

P  +  N 

Verbal 

Quan. 

OTS  w/PPL 

25 

10 

50 

15 

10 

OTS  w/o  PPL 

50 

50 

60 

15 

10 

ROTC  (pre-1997) 

50 

10 

60 

15 

30 

ROTC  (1997  &  later) 

25 

10 

50 

15 

10 

Active  Duty 

25 

10 

50 

none 

none 

AFA  (pre-1998) 

none 

none 

none 

none 

none 

Note:  "PPL"  means  private  pilot's  license. 


tional  tests  (5,  6).  Some  experimental  tests  were  re- 
moved  during  battery  development  for  psychometric 
reasons  (e.g.,  lack  of  incremental  validity).  When  the 
BAT  was  operationally  implemented  in  1993,  the  Men¬ 
tal  Rotation  test  was  still  administered,  but  was  not 
scored.  The  BAT  was  operationally  implemented  for 
pilot  selection  in  1993.  Scores  from  the  BAT  are  com¬ 
bined  with  the  AFOQT  Pilot  composite  and  a  measure 
of  flying  experience  to  produce  a  pilot  aptitude  com¬ 
posite  known  as  PCSM  (Pilot  Candidate  Selection 
Method;  6,  8).  PCSM  scores  have  been  shown  to  be 
related  to  probability  of  completing  UPT  (6,7),  number 
of  flying  hours  needed  to  complete  training  (11),  class 
ranking  (7),  and  fighter  qualification  (20).  Higher  PCSM 
scores  are  associated  with  greater  probability  of  com¬ 
pleting  jet  training,  fewer  hours  needed  to  complete 
training,  higher  class  rank,  and  greater  likelihood  of 
being  fighter  qualified. 

Pilot  Selection  Standards 

As  illustrated  in  Table  I,  the  minimum  qualifying 
AFOQT  scores  for  pilot  selection  vary  by  commission¬ 
ing  source.  The  strictest  standards  are  for  OTS  pilot 
applicants  without  a  private  pilot's  license  (PPL).  The 
weakest  standards  are  for  AFA  pilot  applicants.  The 
AFA  sets  no  minimum  qualifying  AFOQT  score,  in¬ 
stead  using  it  as  a  "tie  breaker"  when  evaluating  pilot 
applicants  (19)*.  There  are  no  established  minimum 
qualifying  scores  for  the  BAT  or  PCSM. 

There  are  seven  major  points  of  entry  into  USAF  pilot 
training  (19).  These  include  the  Air  Force  Academy 
(AFA),  Active  Duty  (AD),  Officer  Training  School 
(OTS),  Reserve  Officer  Training  Corps  (ROTC),  Air  Na¬ 
tional  Guard  (ANG),  Air  Force  Reserves,  and  interna¬ 
tional  pilots.  This  paper  will  focus  on  the  first  four,  as 
they  are  the  most  common  sources.  Weeks  and  Zelenski 
(19)  provide  a  more  detailed  description. 

Air  Force  Academy  (AFA):  The  AFA  requires  interested 
cadets  to  complete  flight  screening  for  qualification 
prior  to  meeting  a  selection  board.  The  AFA  pilot  selec¬ 
tion  board  consists  of  six  or  seven  AFA  faculty  mem- 


*The  AFOQT  is  not  used  by  the  AFA  for  either  officer  commission¬ 
ing  or  flying  training  qualification.  During  the  period  when  the  pilot 
selection  board  policy  capturing  study  (18,  19)  was  done,  flymg 
training  qualified  AFA  students  took  the  AFOQT  on  an  experimental 
basis. 


bers  and  a  non- AFA  representative.  All  board  members 
are  pilots. 

Applicant  selection  folders  contain:  a)  the  Cadet  Ad¬ 
ministration  Management  Information  System  (CAMIS) 
sheet;  b)  evaluations  by  AFA  faculty  members;  and  c)  an 
optional  letter  by  the  pilot  candidate.  The  CAMIS  sheet 
provides  detailed  information  on:  personnel  data;  aca¬ 
demic,  military,  and  athletic  performance;  honors;  flight 
screening  performance;  participation  in  airmanship  and 
aviation  courses;  involvement  in  clubs;  probation  history; 
and  AFOQT  scores  (not  required  for  qualification).  No 
explicit  gender  or  race/ethnicity  information  is  provided. 

Active  Duty  (AD):  Active  duty  officers  may  apply  for 
pilot  training  through  the  Air  Force  Personnel  Center 
(AFPC).  Qualification  is  based  on  age,  medical,  and 
anthropometric  standards,  and  minimum  qualifying 
AFOQT  scores.  The  AD  pilot  selection  board  consists  of 
the  AFPC  Vice  Commander,  an  AFPC  colonel,  and  two 
lieutenant  colonels  that  are  current  or  former  flying 
squadron  commanders. 

The  AD  selection  board  process  is  similar  to  that  used 
by  the  AFA  and  OTS.  Board  members  are  instructed  to 
use  a  "whole  person"  concept.  The  relative  importance 
of  selection  factors  is  not  prescribed,  but  is  left  up  to  the 
individual  board  members.  Board  members  attempt  to 
reach  a  consensus  on  the  evaluation  of  each  record. 
When  divergent  ratings  occur,  they  are  resolved  by 
rescoring  applicants'  records. 

Application  folders  contain  an  Officer  Application  for 
Flying  Training,  Officer  Performance  Reports  (OPRs), 
training  reports,  citations  accompanying  military  deco¬ 
rations,  a  career  brief,  and  the  PCSM  score.  Unlike  the 
AFA,  AD  selection  folders  contain  explicit  information 
about  applicants'  gender  and  race/ethnicity. 

Ojficer  Training  School  (OTS):  As  a  group,  OTS  pilot 
applicants  tend  to  be  older  than  other  USAF  pilot  ap¬ 
plicants,  are  college  graduates,  and  may  have  signifi¬ 
cant  civilian  flying  experience.  Many  have  prior  mili¬ 
tary  experience,  typically  as  active  duty  Air  Force 
enlisted  members.  OTS  pilot  selection  boards  consist  of 
three  pilots  (colonels  or  colonel-selects). 

Compared  with  other  commissioning  sources,  OTS 
pilot  applicant  folders  contain  the  most  comprehensive 
set  of  information.  As  with  the  AFA  and  Active  Duty, 
board  members  are  instructed  to  use  a  "whole  person" 
approach.  Board  members  are  provided  a  scoring  pol¬ 
icy  that  suggests  the  relative  importance  of  three  broad 
categories  of  information:  a)  experience  (civilian  or  mil¬ 
itary  employment,  outside  activities;  0-3  points);  b) 
education  and  aptitude  (academic  history,  AFOQT, 
PCSM;  0-3  points);  and  c)  potential  and  adaptability 
(recommendations;  0-4  points).  As  with  Active  Duty 
boards,  divergent  ratings  are  resolved  by  rescoring  ap¬ 
plicant  records. 

Selection  folders  include  aptitude  test  scores  (i.e., 
AFOQT  and  PCSM),  biographical  and  educational  in¬ 
formation,  employment  history,  history  of  legal  viola¬ 
tions,  civilian  flying  records  including  flying  and  FAA 
medical  certificates,  letters  of  recommendation,  and 
other  data.  As  with  Active  Duty  selection  procedures, 
OTS  selection  boards  are  provided  explicit  information 
about  candidates'  gender  and  race/ethnicity. 
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Reserve  Officer  Training  Corps  (ROTC):  ROTC  pilot 
applicants  are  cadets  in  their  junior  year  of  college. 
There  are  nearly  150  college  and  iiniversity  ROTC  de¬ 
tachments  that  can  provide  applicants.  A  centralized 
board  consisting  of  rated  field  grade  officers  from  HQ 
AFROTC  at  Maxwell  AFB,  AL,  makes  pilot  candidate 
selection  decisions. 

The  process  of  categorizing  ROTC  pilot  applicants  is 
different  from  that  used  by  the  other  commissioning 
sources.  Board  members  are  not  required  to  manually 
review  the  detailed  contents  of  folders.  The  ROTC  pilot 
selection  process  is  supported  by  preprocessing  of  ap¬ 
plicant  data  to  create  a  preliminary  rank-order  of  appli¬ 
cants.  As  a  result,  the  workload  for  ROTC  board  mem¬ 
bers  is  minimized  and  the  relative  importance  of 
selection  factors  is  the  same  for  all  applicants. 

The  algorithm  that  supports  ROTC  pilot  candidate 
selection  weighs  selection  factors  to  create  a  composite 
known  as  the  Categorization  Order  of  Merit  (COM) 
score.  The  information  that  contributes  to  the  COM 
score  includes  an  evaluation  of  the  cadet's  officer  po¬ 
tential  by  the  detachment  commander  (i.e..  Relative 
Standing  Score’*’,  college  grade  point  average,  AFOQT 
composite  scores,  and  a  physical  fitness  test  score).  It  is 
interesting  to  note  that  prior  civilian  flying  experience 
does  not  contribute  to  the  COM  score.  Prior  to  1998,  the 
PCSM  score  also  did  not  contribute  to  the  COM  score. 
The  ROTC  COM  equation  was  modified  (14)  in  1998, 
and  the  AFOQT  Pilot  composite  was  removed  and  re¬ 
placed  by  the  PCSM  composite. 

Cadets  are  rank-ordered  from  highest  to  lowest  based 
solely  on  COM  score.  The  pilot  production  requirement 
is  applied  to  produce  an  initial  cut  line.  The  selection 
board  then  reviews  the  list  considering  all  information 
presented.  It  may  accept  the  rank-ordered  list  as  pre¬ 
sented  or  adjust  the  ordering  on  the  basis  of  selection 
criteria  independent  of  the  COM  score  (e.g.,  ROTC  de¬ 
tachment,  gender,  race /ethnicity,  previous  flying  expe¬ 
rience,  field  training  awards,  academic  major). 

Selection  Ratios 

The  proportion  of  applicants  selected  for  pilot  train¬ 
ing  varies  by  point  of  entry  and  as  a  function  of  the 
needs  of  the  U.S.  Air  Force  (i.e.,  annual  pilot  production 
quotas).  Over  the  last  several  years,  most  pilot  candi¬ 
dates  have  come  from  the  AFA  and  ROTC.  In  contrast, 
OTS  selection  boards  have  been  very  selective  during  a 
period  of  low  pilot  production.  For  example,  in  1996  the 
pilot  selection  ratios  were  0.789  (375/475)  for  the  AFA, 
0.147  (81/551)  for  Active  Duty,  0.047  (16/340)  for  OTS, 
and  0.711  (382/537)  for  ROTC  (16).  The  combined  se¬ 
lection  ratio  for  all  four  sources  was  0.449  (854/1903). 


^Detachment  commanders  are  instructed  to  employ  a  "whole  per¬ 
son"  concept  when  evaluating  officer  potential.  Each  cadet  is  rank- 
ordered  relative  to  all  other  cadets  in  the  detachment.  Detachment 
rankings  are  then  adjusted  to  take  into  accoimt  differences  in  class 
size  across  detachments.  The  adjusted  value  is  expressed  as  a  Relative 
Standing  Score  (RSS). 


USAF  Pilot  Selection  Board  Policy  Capturing  Study 

The  USAF  recently  conducted  a  policy  capturing  ex¬ 
ercise  to  better  understand  the  process  by  which  the 
AFA,  Active  Duty,  OTS,  and  ROTC  pilot  selection 
boards  made  their  decisions  (18,19).  Policy  capturing 
involves  quantifying  the  process  used  by  a  decision¬ 
maker  through  which  informational  attributes  are 
weighed  and  combined  to  make  a  decision.  Results  for 
Active  Duty  and  OTS  boards  should  be  viewed  as  pre¬ 
liminary  due  to  the  small  number  of  pilots  produced 
from  those  commissioning  sources. 

The  policy  capturing  method  used  was  the  same  for 
each  point  of  entry.  The  only  differences  were  in  the 
information  available  in  the  applicants'  folders  and  the 
time  frame  for  the  data  collection.  So,  rather  than  de¬ 
scribing  the  policy  capturing  method  separately  for 
each,  it  is  described  only  for  the  AFA.  The  AFA  pro¬ 
vided  applicant  folders  and  average  board  ratings  for 
the  1995,  1996,  and  1997  classes.  Research  team  mem¬ 
bers  observed  the  selection  board  process  and  surveyed 
board  members  to  identify  the  most  important  at¬ 
tributes.  A  database  was  constructed  and  a  series  of 
linear  regression  analyses  were  performed  to  determine 
which  applicant  variables  were  actually  used  by  selec¬ 
tion  boards.  Applicant  information  was  used  as  the 
independent  variables  and  average  board  rating  was 
the  dependent  variable.  Using  input  from  selection 
board  members,  relevant  applicant  attributes  were  se¬ 
lected  and  included  in  a  preliminary  'baseline"  selec¬ 
tion  policy  model.  Variables  that  were  highly  signifi¬ 
cantly  correlated  with  average  board  rating  were 
retained  and  those  that  were  not  were  discarded.  Once 
a  model  was  determined  for  a  specific  board  for  a 
specific  year,  it  was  applied  to  the  following  yearly 
boards  to  evaluate  the  stability  of  the  policy  model 
across  boards.  For  the  AFA,  selection  policy  models 
were  developed  for  1995, 1996,  and  1997,  and  a  general 
model  was  developed  for  all  years  combined. 

The  yearly  AFA  pilot  selection  board  policy  captur¬ 
ing  models  showed  great  consistency  from  year  to  year, 
with  no  statistically  significant  differences  between  the 
separate  yearly  models.  The  general  model  (across 
years)  showed  a  very  good  fit,  accounting  for  90%  of  the 
variability  in  board  ratings  (R^  =  0.90).  Results  indi¬ 
cated  a  heavy  reliance  on  cumulative  military  perfor¬ 
mance  average,  cumulative  grade  point  average,  and 
flight  screening  performance.  Variables  that  contrib¬ 
uted  to  a  lesser  extent  to  the  AFA  policy  capturing 
model  were:  participation  in  a  varsity  sport,  whether  or 
not  the  applicant  had  held  a  commander  military  posi¬ 
tion,  and  whether  or  not  the  cadet  had  held  a  lower 
military  position  in  their  senior  year.  "Officership"  as 
indicated  by  military  performance  was  more  important 
for  pilot  selection  decisions  than  was  ability  as  mea¬ 
sured  by  cumulative  grade  point  average  and  flight 
screening  performance  rating.  AFOQT  scores  were 
given  little  importance  and  were  used  mainly  as  a  "tie 
breaker"  for  distinguishing  among  candidates  close  to 
the  "cut  line." 

For  Active  Duty  pilot  selection  boards,  separate  pol¬ 
icy  capturing  analyses  were  done  for  non-rated  line 
officers  and  navigators  for  fiscal  years  1996  and  1997.  In 
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each  case,  the  resultant  general  (combined  fiscal  years) 
policy  capturing  model  showed  a  good  fit  (R^  =  0.78 
and  0.72,  respectively).  For  non-rated  line  officers,  the 
most  important  selection  factor  by  far  was  PCSM.  Vari¬ 
ables  that  contributed  to  a  lesser  extent  to  the  non-rated 
model  were:  2nd  lieutenant  rank,  positive  endorser's 
recommendation,  possession  of  an  engineering  or 
mathematics  degree,  possession  of  a  master's  degree, 
and  possession  of  a  flying  instruments  rating. 

For  Active  Duty  navigators,  the  most  important  se¬ 
lection  factors  were  PCSM  and  the  AFOQT  Quantita¬ 
tive  composite.  Positive  endorser's  recommendation 
and  positive  commander's  recommendation  contrib¬ 
uted  to  a  lesser  extent. 

OTS  selection  boards  provided  applicant  folders  and 
average  board  ratings  for  the  first  and  second  quarters 
of  fiscal  year  1996.  The  general  (combined  quarters) 
OTS  pilot  selection  board  policy  capturing  model 
showed  a  relatively  poor  fit  (R^  =  0.51)  compared  with 
results  from  the  other  points  of  entry.  The  relatively 
poor  fit  was  probably  due  to  the  small  number  of  ap¬ 
plicants  selected.  The  most  important  factors  were  col¬ 
lege  grade  point  average  and  PCSM  score.  Number  of 
recommendation  letters,  interviewers'  recommenda¬ 
tions,  number  of  moving  traffic  violations,  and  posses¬ 
sion  of  a  bachelor  of  arts  degree  contributed  to  a  much 
lesser  extent  to  the  OTS  policy  capturing  model. 

For  the  ROTC,  the  COM  score  was  by  far  the  most 
important  factor  in  pilot  candidate  selection.  Very  few 
candidates  were  re-ordered  on  the  basis  of  non-COM 
data.  As  a  result,  no  formal  policy  capturing  study  was 
conducted  (18,19).  Instead,  analyses  focused  on  imder- 
standing  the  relative  importance  of  the  COM  compo¬ 
nents  in  determining  the  overall  COM  score.  Examina¬ 
tion  of  the  weights  and  upper  and  lower  bounds  of  the 
COM  components  revealed  that  the  commander's  as¬ 
sessment  of  the  cadet's  officer  potential  (Relative  Stand¬ 
ing  Score  or  RSS)  accounted  for  about  47.8%  of  the 
variability  in  the  COM  score.  The  other  COM  compo¬ 
nents  accounted  for  the  remaining  52.2%  as  follows: 
college  grade  point  average  (19.8%),  physical  fitness  test 
score  (11.5%),  AFOQT  Verbal  composite  (11.5%),  and 
AFOQT  Quantitative  composite  (9.4%). 

Examination  of  Pearson  correlation  coefficients  be¬ 
tween  the  COM  score  and  its  components  provided 
another  way  of  looking  at  the  relative  importance  of  the 
COM  components  (18,19)  in  determining  overall  COM 
score.  As  in  the  analysis  of  the  COM  weights,  RSS  had 
the  strongest  relationship  with  overall  COM  score.  The 
correlations  between  COM  and  its  components  were: 
RSS  (0.921),  college  grade  point  average  (0.639),  physi¬ 
cal  fitness  test  score  (0.437),  AFOQT  Verbal  composite 
(0.290),  and  AFOQT  Quantitative  composite  (0.325). 
These  results  taken  together  clearly  showed  that  RSS 
was  the  most  influential  determinant  of  the  overall 
COM  score,  and  as  a  result,  ROTC  pilot  selection  deci¬ 
sions.  Pilot  aptitude  measures  such  as  PCSM  or  previ¬ 
ous  flying  experience  were  virtually  ignored  in  the 
ROTC  pilot  selection  process. 

The  two  largest  sources  of  USAF  pilots  (AFA  and 
ROTC)  place  a  much  greater  emphasis  on  measures  of 
officership  (AFA — military  performance  average. 


ROTC — RSS)  than  measures  of  ability  (e.g.,  academic 
performance,  flight  screening  performance,  aptitude 
test  scores)  when  selecting  pilot  trainees.  This  is  rather 
perplexing  in  the  absence  of  studies  showing  validity 
for  measures  of  officership  for  predicting  pilot  training 
performance. 

Weeks  (18)  reported  a  study  of  the  relative  predictive 
utility  of  officership  as  measured  by  the  RSS  and  ability 
as  measured  by  the  AFOQT  Pilot  composite  for  469 
ROTC  cadets  who  attended  imdergraduate  pilot  train¬ 
ing.  The  correlation  between  RSS  and  pass /fail  training 
was  0.01  (ns),  whereas  the  correlation  between  AFOQT 
pilot  composite  and  pass/fail  training  was  0.14  (p  < 
0.01).  These  analyses  indicate  almost  no  relationship  be¬ 
tween  officership  ratings  and  pilot  training  attrition. 
The  strength  of  the  relationship  between  ability  and 
pilot  training  outcome  is  lower  than  usually  reported 
(see  for  example,  8, 9),  but  still  clearly  shows  that  ability 
represents  an  improvement  over  officership  ratings  for 
selection. 

Validity  of  PCSM  for  T-37  Training 

Operational  PCSM  testing  began  in  June  1993  and 
selection  boards  first  received  PCSM  scores  in  Decem¬ 
ber  1993.  Air  Education  and  Training  Command  Stud¬ 
ies  and  Analyses  Squadron  (AETC/SAS)  subsequently 
conducted  an  evaluation  of  PCSM  in  June  1997  (16).  At 
the  time  of  the  evaluation,  7057  USAF  pilot  training 
applicants  had  valid  PCSM  scores  (had  tested  on  the 
AFOQT  and  BAT).  Of  these,  1276  had  been  selected  and 
entered  pilot  training;  676  either  had  been  eliminated 
from  training  or  had  successfully  completed  T-37  train¬ 
ing.  The  T-37  graduation  rate  was  81.2%  (549/ 676).  The 
attrition  figures  include  pilot  candidates  who  failed  in 
either  the  Flight  Screening/Enhanced  Flight  Screening 
programs  or  during  the  T-37  phase  of  SUPT.  Ness  (16) 
did  not  report  the  correlation  between  PCSM  and  the 
dichotomous  pass /fail  training  outcome,  but  reported 
the  pass  rate  for  trainees  grouped  by  PCSM  score  (1- 
25,26-50,51-75,  and  76-99).  The  T-37  pass  rates  for  the 
four  groups  were:  1-25:  58.4%  (66/113);  26-50:  69.7% 
(83/119);  51-75:  82.4%  (103/125);  and  76-99:  93.1% 
(297/319).  Ness  (16)  observed  that  pilot  trainees  with 
PCSM  scores  of  50  or  lower  were  much  less  likely  to 
complete  T-37  training  than  were  those  with  scores 
greater  than  50  (64.2%  vs.  90.1%  pass  rate).  Based  on 
tiiese  results,  he  recommended  that  a  minimum  PCSM 
score  of  50  be  adopted  and  that  PCSM  be  extended  to 
the  AFA. 

In  a  further  breakout  of  PCSM  scores  for  the  676  pilot 
trainees,  Ness  (16)  displayed  the  number  of  trainees 
with  scores  of  1-25,  26-50,  51-75,  and  76-99  for  each 
commissioning  source  that  tested  their  pilot  applicants 
on  PCSM  (AD,  OTS,  and  ROTC).  Almost  all  AD  and 
OTS  pilot  trainees  had  PCSM  scores  greater  than  50, 
with  most  scores  greater  than  75.  In  contrast,  ROTC 
pilot  trainees  were  nearly  equally  likely  to  have  scores 
in  any  of  the  four  PCSM  groups  (1-25,26-50,51-75,  and 
76-99).  This  is  consistent  with  the  pilot  selection  board 
policy  capturing  study  results  (18,19)  that  AD  and  OTS 
selection  boards  relied  on  PCSM  in  their  selection  pro¬ 
cess  and  ROTC  boards  did  not. 
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Fig.  2.  T  -37  Pass/Fail  Outcome  by  PCSM  Score. 


A  second  examination  of  the  utility  of  PCSM  was 
conducted  for  this  paper  in  order  to  get  an  estimate  of 
its  validity  against  T-37  outcome.  As  of  Fall  1998,  1268 
USAF  pilot  trainees  with  PCSM  scores  had  completed 
the  T-37  phase  of  SUPT^.  The  T-37  graduation  rate  was 
80.1%  (1016/1268).  As  observed  in  the  AETC/SAS 
PCSM  evaluation  (16),  examination  of  the  PCSM  scores 
for  these  trainees  clearly  showed  that  some  selection 
boards  did  not  use  the  PCSM  score.  The  mean  PCSM 
percentile  score  was  62.8  and  ranged  from  1  to  99.  Some 
229  (18.1%)  of  the  trainees  had  PCSM  scores  of  25  or 
lower,  and  23  (1.8%)  had  PCSM  scores  of  1. 

The  observed  correlation  between  PCSM  and  the  di¬ 
chotomous  pass/fail  T-37  training  criterion  was  0.335. 
The  correlation  increased  to  0.479  after  correction  for 
dichotomization  of  the  training  criterion.  As  shown  in 
Fig.  2,  higher  PCSM  scores  were  clearly  related  to 
greater  probability  of  successfully  completing  T-37 
training.  The  pass  rates  for  the  four  groups  were:  1-25: 
58.5%  (134/229);  26-50:  69.7%  (163/234);  51-75:  81.8% 
(180/220);  and  76-99: 92.1%  (539/585).  As  noted  earlier, 
AD,  OTS,  and  ROTC  currently  set  minimum  qualifying 
AFOQT  Pilot  composite  scores  for  pilot  training  with 
values  of  either  25  or  50  (see  Table  I).  If  a  minimum 
qualifying  PCSM  score  of  25  had  been  used,  the  overall 
T-37  training  pass  rate  would  have  been  about  84.5% 
(882/1039)  instead  of  the  observed  80.1%  (1016/1278). 
Had  a  minimum  qualifying  PCSM  score  of  50  been 
used,  as  suggested  by  Ness  (16),  the  T-37  training  pass 
rate  would  have  been  about  89.3%  (719/805).  Results 
from  both  Ness  (16)  and  the  current  study  clearly  show 
that  establishment  of  a  minimum  PCSM  score  consis¬ 
tent  with  current  AFOQT  minimum  scores  for  pilot 
training  qualification  could  help  reduce  training  attri¬ 
tion. 

Final  Thoughts 

Current  AETC  efforts  are  focusing  on  upgrading  the 
BAT  hardware  and  test  battery.  AETC  has  begun  the 
work  needed  to  replace  the  original  386-based  systems 
with  Pentium-based  systems  and  to  evaluate  candidate 
replacement  BAT  tests. 


*This  sample  included  the  676  pilot  trainees  in  Ness'  (14)  earlier 
analyses. 


Cumulative  research  findings  suggest  that  USAF  pi¬ 
lot  selection  decisions  could  be  improved  simply  by 
making  better  use  of  currently  available  personnel  at¬ 
tribute  data.  Higher  PCSM  scores  are  associated  with 
greater  probability  of  completing  jet  training  (6,7,16), 
fewer  flying  hours  needed  to  complete  training  (11), 
higher  class  rank  (7),  and  greater  likelihood  of  being 
fighter  qualified  (20).  Despite  this,  no  minimum  quali¬ 
fying  PCSM  score  has  been  adopted  to  date.  The  reason 
for  this  is  unclear.  It  appears  that  the  various  commis¬ 
sioning  sources  are  reluctant  to  give  up  any  autonomy 
they  currently  have  in  setting  pilot  selection  policy. 
However,  the  PCSM  program  has  been  extended  until 
2003.  AETC/CC  has  directed  the  use  of  PCSM  in  all 
pilot  candidate  boards.  As  of  March  1999,  all  have 
implemented  PCSM  except  the  AFA.  In  1998,  ROTC 
modified  the  COM  equation  removing  the  AFOQT  Pilot 
composite  and  replacing  it  with  the  PCSM  score  (14). 
The  effectiveness  of  this  modification  for  reducing  pilot 
training  attrition  will  depend  on  the  relative  weight 
assigned  to  the  new  COM  components. 

Results  from  recent  studies  in  personnel  selection  and 
occupational  performance  provide  some  clues  as  to 
how  personnel  attribute  data  can  be  better  integrated 
into  the  pilot  selection  process  (8,15,17).  The  validity  of 
PCSM  has  been  shown  to  come  mostly  from  the  mea¬ 
surement  of  cognitive  ability,  psychomotor  ability,  pilot 
job  knowledge,  and  flying  (i.e.,  job-related)  experience 
(8).  This  is  consistent  with  results  from  a  large-scale 
meta-analysis  of  the  validity  and  utility  of  selection 
methods  for  training  and  job  performance  across  a  va¬ 
riety  of  occupations  (17).  Schmidt  and  Hunter  (17)  es¬ 
timated  the  predictive  validity  of  general  mental  ability 
(GMA)  tests  to  be  0.56^  for  training.  The  two  combina¬ 
tions  of  predictors  with  the  highest  multivariate  valid¬ 
ity  for  job  training  programs  were  GMA  plus  an  integ¬ 
rity  test  (mean  validity  of  0.67)  and  GMA  plus  a 
conscientiousness  test  (mean  validity  of  0.65).  For  job 
performance,  the  predictive  validity  of  GMA  tests  was 
estimated  to  be  0.51  and  was  incremented  by  the  addi¬ 
tion  of  a  work  sample  test  (0.63),  a  structured  interview 
(0.63),  an  integrity  test  (0.65),  or  a  conscientiousness  test 
(0.60).  Similarly,  McHenry,  et  al.  (15)  reported  that  mea¬ 
sures  of  temperament/personality  were  incremental  to 
cognitive  measures  for  predicting  U.S.  Army  training 
criteria  reflecting  "effort  and  leadership,"  "personal  dis¬ 
cipline,"  and  "physical  fitness  and  military  bearing." 
McHenry  et  al.'s  (15)  results  suggest  that  measures  of 
temperament/personality  might  be  useful  for  officer 
selection,  which  is  closely  tied  to  pilot  selection  in  the 
USAF,  as  pilot  applicants  must  first  be  accepted  for 
officer  commissioning  (19). 

To  reiterate,  the  effective  use  of  currently  available 
personnel  attribute  data  could  lead  to  cost  avoidance 
savings  due  to  reduced  training  attrition  (16)  and  the 
requirement  for  fewer  hands-on  flying  training  hours 


^Schmidt,  and  Hunter  (17)  corrected  their  validity  estimates  for 
measurement  error  in  the  training  or  job  performance  measures.  The 
validity  estimates  were  also  corrected  for  range  restriction  on  the 
selection  method  in  incumbent  samples  relative  to  applicant  popula¬ 
tions. 
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(11).  Results  from  studies  of  personnel  selection  and 
occupational  performance  (15,17)  suggest  that  the  va¬ 
lidity  and  utility  of  current  USAF  pilot  selection  meth¬ 
ods  (i.e.,  selection  boards,  PCSM)  could  be  enhanced  by 
the  introduction  of  a  structured  selection  interview  and 
measures  of  personality. 
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